Characterizing intra-tumoral heterogeneity for response and outcome prediction using radiomic spatial textural descriptor (radistat)

ABSTRACT

Embodiments access an image of a region of interest (ROI) demonstrating cancerous pathology; extract radiomic features from the ROI; define a radiomic feature expression scene based on the ROI and radiomic features; generate a cluster map by superpixel clustering the expression scene; generate an expression map by repartitioning the cluster map into expression levels; compute a textural and spatial phenotypes for the expression map based on the expression levels; construct a radiomic spatial textural (RADISTAT) descriptor by concatenating the textural and spatial phenotypes; provide the RADISTAT descriptor to a machine learning classifier; receive, from the machine learning classifier, a first probability that the ROI is a responder or non-responder, or a second probability that the ROI will experience long-term survival or short-term survival, based, at least in part, on the RADISTAT descriptor; and generate a classification of the ROI as a responder or non-responder, or long-term survivor or short-term survivor.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application62/509,987, filed May 23, 2017, which is incorporated herein in itsentirety.

FEDERAL FUNDING NOTICE

This invention was made with government support under grants1U24CA199374-01, R01CA202752-01A1, R01CA208236-01A1, R21CA179327-01,R21CA195152-01, R01DK098503-02, and 1 C06 RR012463-01 awarded by theNational Institutes of Health. Also grants W81XWH-13-1-0418,W81XWH-14-1-0323, and W81XWH16-1-0329 awarded by the Department ofDefense. The government has certain rights in the invention.

BACKGROUND

Radiomic analysis typically involves extracting a series of quantitativefeatures from a target tissue region of interest (ROI) via radiographicimaging. The target ROI is then described via statistics of the radiomicfeature distribution (e.g. mean, skewness, kurtosis), which are theninput to a machine learning classifier to make a class label prediction.Radiomics has been employed for prediction of disease aggressiveness andsubtype in vivo, as well as characterizing molecular heterogeneity oftumors. However, existing approaches to radiomics-based prediction thatemploy statistical descriptors may not adequately capture the diversityof radiomic expression present in the target ROI, thus incompletelycharacterizing the underlying tissue heterogeneity.

Tumor environment heterogeneity on radiographic imaging arises due tothe organization of multiple tissue pathologies or sub-compartments. Forexample, in Glioblastoma multiforme (GBM), the tumor region includesvaried tissue types such as edema, necrotic core, and enhancing tumor.Similarly, in rectal cancer (RCa) patients that undergo neoadjuvantchemoradiation therapy, treatment effects such as fibrosis andulceration are present both within and proximal to the tumor region. Asa result of such significant tissue heterogeneity, the resultingradiomic response within and around these tumors appears highly varied,as illustrated in FIG. 1 at, which illustrate representative radiomicheatmaps 110 and 112 in RCa. Thus, existing approaches which utilizeconventional statistics such as the mean or skewness value of thesefeature distributions may not adequately describe the diverse radiomicexpression map exhibited by different disease subtypes. Existingapproaches may therefore be sub-optimal in predicting outcomes orcharacterizing response to treatment. Consequently, there is a clinicalunmet need for a more comprehensive descriptor of the organization ofradiomic expression for disease characterization via radiographicimaging.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate various example apparatus,methods, and other example embodiments of various aspects of theinvention. It will be appreciated that the illustrated elementboundaries (e.g., boxes, groups of boxes, or other shapes) in thefigures represent one example of the boundaries. One of ordinary skillin the art will appreciate that, in some examples, one element may bedesigned as multiple elements or that multiple elements may be designedas one element. In some examples, an element shown as an internalcomponent of another element may be implemented as an external componentand vice versa. Furthermore, elements may not be drawn to scale.

FIG. 1 illustrates representative radiomic feature maps for differentRCa patients.

FIG. 2 illustrates cluster maps for different RCa patients.

FIG. 3 illustrates textural and spatial phenotypes between different RCapatients.

FIG. 4 is a schematic overview of an exemplary workflow for obtaining aradiomic spatial textural descriptor.

FIG. 5 illustrates representative radiomic feature maps for differentGBM patients.

FIG. 6 illustrates cluster maps for different GBM patients.

FIG. 7 illustrates textural and spatial phenotypes between different GBMpatients.

FIG. 8 illustrates graphs comparing performance in distinguishingresponse to treatment or predicting survival time in RCa or GBMpatients.

FIG. 9 is a flow diagram of example operations for classifying a regionof interest as a responder or non-responder, or as a long-term survivoror short-term survivor.

FIG. 10 is a flow diagram of example operations for classifying a regionof interest as a responder or non-responder, or as a long-term survivoror short-term survivor.

FIG. 11 illustrates an example apparatus for classifying a region ofinterest as a responder or non-responder, or as a long-term survivor orshort-term survivor.

FIG. 12 illustrates an example apparatus for classifying a region ofinterest as a responder or non-responder, or as a long-term survivor orshort-term survivor.

FIG. 13 illustrates an example computer in which example embodimentsdescribed herein may operate.

FIG. 14 illustrates an example method for classifying a region ofinterest as a responder or non-responder, or as a long-term survivor orshort-term survivor.

DETAILED DESCRIPTION

Embodiments described herein distinguish treatment response in patientsdemonstrating cancerous pathology, including RCa and GBM. Embodimentsdescribed herein also predict overall survival in patients demonstratingcancerous pathology, including RCa and GBM. Embodiments define andemploy a Radiomic Spatial Textural Descriptor (RADISTAT) that ascribes acommon textural and spatial phenotype to a radiomic feature expressionmap within a target region to characterize tissue heterogeneity.Embodiments characterize radiomic phenotypes for at least two clinicallychallenging problems: (a) distinguishing favorable from poor treatmentresponse in patients demonstrating cancerous pathology, including RCa,and (b) predicting survival status in patients demonstrating cancerouspathology, including GBM. Embodiments significantly outperform existingstatistical approaches for predicting overall survival or treatmentresponse. Embodiments may distinguish treatment response or predictoverall survival in patients demonstrating cancerous pathology based ona region of interest (ROI) represented in radiological imagery,including T2-w magnetic resonance imaging (MRI) images, T1-w MRI images,or diffusion-weighted MRI images, including multiple contrast enhancedMRI images.

Radiographic heterogeneity of tumors is often caused by the presence ofmultiple tissue classes being present within and immediately proximal tothe tumor. Multiple tumor classes may include, for example, edema,necrosis, or fibrosis. This radiographic tumor heterogeneity is mirroredin the variable radiomic feature responses, such as regions ofover-expression or under-expression, in such tumors. Radiomic analysiswithin the tumor may involve use of statistical descriptors of thefeature distribution (e.g. mean, skewness, or kurtosis), which are theninput to a predictive model (e.g. machine learning classifier). However,a single statistic may not fully capture the rich, spatial diversity ofradiomic expression within a tissue region. In digital pathology, thespatial architecture of histologic primitives may be predictive ofdisease outcome. Embodiments described herein capture the spatialarrangement of differential radiomic expression within a target area orROI and are more predictive of disease outcome than statistics of theradiomic feature distribution employed in existing approaches.

Embodiments described herein employ a RADISTAT descriptor which (a) morecompletely characterizes the spatial diversity of hot (over-expression)and cold spots (under-expression) exhibited by a radiomic feature, and(b) captures the overall textural appearance of a radiomic feature basedon the relative abundance of hot and cold spots. Embodiments mayfacilitate, for example, (a) discriminating favorable from un-favorabletreatment response in RCa patients, and (b) distinguishing short-termfrom long-term survivors in GBM patients. Embodiments significantlyimprove classification performance (AUC=0.79 in RCa, AUC=0.75 in GBM) ascompared to existing approaches that use simple statistics (e.g., mean,variance, skewness, or kurtosis) to describe radiomic co-occurrencefeatures.

In digital pathology, spatial statistics and graph-based features forcapturing the arrangement of primitives (e.g. glands or nuclei) orsub-compartments (e.g. stroma or epithelium) may be predictive ofdisease outcome. Embodiments described herein employ the spatialarrangement of hot and cold spots in terms of radiomic expression tofacilitate improved disease characterization compared to existingapproaches that employ a single first order statistic. In FIG. 2,representative hot sub-compartments 216, cold sub-compartments 212, andmedium sub-compartments 214 on radiomic expression maps 210 and 220,based on quantizing the image into 3 expression levels (e.g., hot,medium, low), are illustrated.

Embodiments described herein employ the RADISTAT approach to quantifyingspatial arrangement of radiomic feature expression to better describetissue heterogeneity. Embodiments capture (a) the spatial phenotype ofradiomic expression, i.e. how sub-compartments of low and high radiomicexpression are spatially located relative to one another within the ROI,and (b) the textural phenotype associated with radiomic expression, i.e.whether an ROI exhibits a predominance of low or high expressionsub-compartments. Embodiments provide improved utility and performancecompared to existing approaches in the context of at least two problems.First, embodiments provide improved evaluation of response tochemoradiation in RCa, by distinguishing favorable response (nometastatic nodes or distant metastasis present after treatment) frompoor response, via post-treatment MRI. Second, embodiments provideimproved differentiation of long-term from short-term survivors withGBM, using treatment-naive MRIs. Embodiments may be employed to predicttreatment response or overall survival time in patients demonstratingother, different cancerous pathologies.

Some existing approaches to analyzing GBM look at separate tumorsub-compartments, albeit using volumetric or radiomic histogram analysisalone. Similarly, sub-compartment-based radiomic analysis of breast MRIand lung fluorodesoxyglucose (FDG) positron emission tomography (PET)/CThas been employed for predicting patient response to treatment as wellas patient survival. In one existing approach, a Gaussian mixture modelof multi-parametric MRI intensities is employed to definesub-compartments in GBMs. Spatial point pattern analysis is then used toperform a neighborhood analysis of these sub-compartments.

In contrast to such existing approaches, embodiments described hereinthat employ RADISTAT generate a more detailed radiomic characterizationof tissue heterogeneity, compared to existing approaches that use MRintensities alone. In contrast to existing approaches, embodimentsdescribed herein leverage computer-extracted radiomic expression maps togenerate intra-tumoral clusters, rather than using MRI intensities priorto any analysis of the sub-compartment distributions. Embodimentsgenerate a spatial characterization of radiomic expression andproportion of different expression levels. Embodiments definesub-compartments on the radiomic feature expression map through a2-stage process: (1) superpixel clustering of the radiomic feature toidentify spatially similar regions, and (2) re-partitioning thesuperpixel map to define sub-compartments based to a desired number ofexpression levels (e.g. over-expression (hot), medium expression (warm),and under-expression (cold), when considering 3 expression levels).Finally, embodiments employing RADISTAT compute at least two distinctfeatures: (1) the overall spatial arrangement of differentsub-compartments with respect to one another, and (2) the overallproportions of different expression levels for the radiomic feature.

FIG. 4 illustrates one exemplary approach for computation of theRADISTAT descriptor. A heatmap 410 of an input radiomic scene I isaccessed. At 420, super-pixel clustering is applied to yield a clustermap Î (grey scale represents dominant clusters). At 430, byre-partitioning cluster map Î into B=3 expression levels, the expressionmap Ĩ is obtained, where each of the three grey levels in this examplerepresents a different expression level. Phenotypes are computed at 440.In this example, a textural phenotype T is computed as the proportion ofeach of the low (L), medium (M), and high (H) expression levels in Ĩillustrated in the top bar plot 442. At the bottom bar plot 444, aspatial phenotype

is computed as the number of times L-M, M-H, and L-H expression levelsare adjacent to each other, based on the adjacency graph depicted by thearrows 432 on Ĩ. The RADISTAT descriptor is then obtained as theconcatenation of τ and

.

The approach illustrated in FIG. 4 is now described in more detail. Inone embodiment, a radiomic feature expression scene is denoted I=(C; f)at 410. In this embodiment, C is a spatial grid of pixels c, in

² or

³. In this embodiment, every pixel, c∈C, is associated with a radiomicfeature value f(c). In another embodiment, less than every pixel isassociated with a radiomic feature value. For example, some pixels maybe associated with imaging artifacts, outliers, or other imperfectionsin the image and thus may not have an associated radiomic feature value,or may be excluded from the radiomic feature expression scene I. Therange of I is normalized to lie between 0 and 1.

At 420, superpixel clustering of the radiomic feature maps is performed.Superpixel clustering of I is performed using a modified version of thesimple linear iterative clustering (SLIC) algorithm, to generate Kclusters, Ĉ_(k)⊂C,k∈{1, . . . , K}. Note that in the modified SLICapproach described herein, K is implicitly defined based on 2parameters: (1) the minimum number of pixels in a cluster (α), and (2)the number of pixels between initial cluster seeds (β). Thus for eachcombination of α and β, different clusterings of I will be obtained.Based on superpixel clustering, I is quantized to obtain a cluster mapÎ=(C,g), where for every c∈Ĉ_(k)⊂C, g(c) is the average radiomic featurevalue within the cluster Ĉ_(k). Note that I is normalized such thatmin(g(c))=0 and max(g(c))=1.

At 430, the superpixel clusters are re-partitioned into expressionlevels. Firstly, a user-defined parameter B, which captures the desirednumber of expression levels, is identified. The choice of B dictates howfine a variation in radiomic feature values is captured by RADISTAT.Using this input parameter B, the range of Î is split into B equallyspaced bins, yielding B+1 thresholds θ_(j),j∈{0, . . . , B}. Based onthe normalized range of Î, θ₀=0 and θ_(B)=1. These θ_(j),j∈{0, . . . ,B}, are used to re-quantize Î into an expression map, {tilde over ( )}I=(C,h), where ∀c∈C, h(c)=θ_(j), if θ_(j-1)<g(c)<θ_(j). As Ĩ only has Bunique values, any adjacent clusters which exhibit the same expressionvalue are merged to yield M distinct partitions. A partition is definedas {tilde over (C)}_(m)={c|h(c)=θ_(j), where m∈{1, . . . , M} and {tildeover (C)}_(m)=C. For ease of notation, we also define the expressionvalue of a partition {tilde over (C)}_(m) as H({tilde over(C)}_(m))=θ_(j), if ∀c∈{tilde over (C)}_(m), h(c)=θ_(j).

For example, when B=3 (corresponding to low, medium, and highexpression), the thresholds θ_(j)={0, 0.33, 0.67, 1}. The resultingexpression map Ĩ will only have three unique values, {0.33, 0.67, 1} butcan have M distinct partitions, as multiple partitions {tilde over(C)}_(m) can have the same expression value.

At 440, phenotypes are computed. The textural phenotype is computed at442, and the spatial phenotype is computed at 444. The texturalphenotype is obtained by quantifying the fraction of each of Bexpression levels in Ĩ. For B=3, this means calculating what fraction ofexpression map Ĩ exhibits low, medium, or high expression. For eachexpression level θ_(j) and ∀j={1 . . . , B},

$\begin{matrix}{\tau_{j} = \frac{{\left. c \middle| {h(c)} \right. = \theta_{j}}}{C}} & \left( {{Eq}.\mspace{14mu} 1} \right)\end{matrix}$

The resulting feature vector is a 1×B vector τ=[τ₁, . . . , τ_(B)].

The spatial phenotype is obtained by quantifying the adjacency for eachpairwise combination of B expression levels in Ĩ. Considering the caseof low (L), medium (M), and high (H) expression (i.e. B=3), there arethree pairwise combinations: L-M, L-H, M-H. The adjacency of L-M isobtained by counting number of times that Ĩ has partitions with low andmedium expression adjacent to each other. The adjacency of L-H isobtained by counting number of times that Ĩ has partitions with low andhigh expression adjacent to each other. The adjacency of M-H is obtainedby counting number of times that Ĩ has partitions with medium and highexpression adjacent to each other. Other adjacencies for other values ofB may be similarly computed.

For this embodiment, an adjacency graph G=(V E) is defined, whereV={v_(m)}, m∈{1, . . . , M}, comprises the centroids of each of Mpartitions obtained at 420; and E={e_(mn)},m,n∈{1, . . . , M} is a setof edges. An edge in E is defined when,

$\begin{matrix}{e_{mn} = \left\{ \begin{matrix}{1,} & {{{if}\mspace{14mu} {\overset{\sim}{C}}_{m}\mspace{14mu} {adjacent}\mspace{14mu} {to}\mspace{14mu} {\overset{\sim}{C}}_{n}},{m \neq n}} \\{0,} & {otherwise}\end{matrix} \right.} & \left( {{Eq}.\mspace{14mu} 2} \right)\end{matrix}$

For every pair of expression levels θ_(i) and θ_(j), i,j∈{1, . . . B}the adjacency is calculated as:

_(mn) =Σe _(mn), where H({tilde over (C)} _(m))=θ_(i) and H({tilde over(C)} _(n))=θ_(j).  (Eq. 3)

The resulting feature vector is a 1×N vector

=[

₁, . . . ,

_(N)], where

$N = \begin{pmatrix}B \\2\end{pmatrix}$

is the total number of expression level pairs in Ĩ. The RADISTATdescriptor is constructed by concatenating τ and

to yield a 1×(B+N) vector.

In summary, FIG. 4 illustrates the workflow of RADISTAT as employed byembodiments described herein, and its implementation in the context ofclinical problems in RCa and GBM. In one embodiment, for each datasetconsidered, a representative two dimensional (2D) section was obtainedfrom the middle of isotropically resampled volumes, and the region ofinterest was annotated by an expert radiologist. Twelve gray levelco-occurrence matrix (GLCM) features were extracted on a pixel-wisebasis from every 2D section, in both datasets. These GLCM features wereentropy, energy, inertia, correlation, information measures 1 and 2, sumand difference averages, variances, and entropies. Embodiments employingRADISTAT were compared against 4 statistical descriptors (mean,variance, skewness, and kurtosis of the radiomic expressiondistribution), τ, and

, for all twelve features. Performance of embodiments employing RADISTATwith a linear discriminant analysis (LDA) classifier in order todifferentiate the two patient groups in each cohort was evaluated. A3-fold cross-validation strategy was employed and the performance ofeach of the radiomic descriptors was compared over different superpixelparameters, α∈{5, 10, 15, 20} and β∈{3, 5, 7}, where α and β are valuesin pixels. In this example, the number of bins was fixed at B=3,corresponding to high, medium, and low radiomic expression levels. α=5and β=7 were empirically found to be optimal parameters across alltwelve GLCM features and were employed for further evaluation.Classifier accuracy was measured as area under the receiver-operativecurve (AUC), with average AUC values over 25 runs of 3-fold crossvalidation for both the cohorts. Kruskal-Wallis multiple comparisontesting was performed to determine statistical significance, based onadjusted p-values via the Bonferroni correction.

Embodiments described herein may be employed to predict overall survivalor distinguish treatment response in patients demonstrating cancerouspathology. For example, one embodiment may predict overall survival timefor a patient demonstrating GBM, while another embodiment maydistinguish treatment response for a patient demonstrating RCa. Anotherembodiment may predict or distinguish treatment response and predictoverall survival for a patient demonstrating cancerous pathologyincluding GBM, RCa, prostate cancer, lung cancer, breast cancer, orother type of cancerous pathology.

Embodiments may compute the RADISTAT descriptor in two dimensions or inthree dimensions. For example, one embodiment that computes the RADISTATdescriptor in two or three dimensions accepts as data the radiomicfeature expression scene I=(C, f). A user inputs a desired number ofexpression levels B. In this embodiment, K clusters in C are generatedsuch that Ĉ_(k)⊂C, k∈{1, . . . , K} using the modified SLIC approach.Then, for each cluster k∈K, the embodiment computes

$\mu_{k} = \frac{\Sigma_{d}{f(d)}}{{\hat{C}}_{k}}$

where d∈Ĉ_(k); and where g(d)=μ_(k), ∀d∈{tilde over (C)}_(k). Then, foreach pixel c∈C, embodiments compute h(c)=θ_(j), if θ_(j-1)<g(c)<θ_(j),where j∈{0, . . . , B}. Adjacent clusters that exhibit the sameexpression values are merged. A textural phenotype expressed as a 1×Bvector τ is obtained as described with respect to Equation 1 above. Aspatial phenotype expressed as a 1×N vector

is obtained as described with respect to Equation 3 above. The RADISTATdescriptor may then be constructed from τ and

.

Returning to FIG. 1, heatmaps 110 and 112 of representative low and highclinically staged patients for RCa following chemotherapy treatment areillustrated. The heatmaps 110 and 112 depict the radiomic featurerepresentation of a single GLCM descriptor, correlation, for each pixel.In this example, higher values of correlation are indicated at 116,while lower values of correlation are indicated at 114. In anotherexample, different values of correlation may be expressed in differentformats, for example, color based heatmaps, or other graphicalrepresentations. Distributions of the radiomic feature expressionbetween the two patients are expressed in graph 120. Distribution ofunfavorable response is illustrated by curve 122, and distribution offavorable response is illustrated by curve 124. Existing approacheswould indicate minimal separation in the distribution curves 122 and 124of the radiomic expression between the two pathologic responsesillustrated in graph 120. Embodiments re-quantize the radiomic heatmaps110 and 112 through superpixel clustering and partitioning asillustrated in FIG. 2 at 210 and 220 respectively, revealing underlyingdifferences in the frequency of binned expression levels that existingapproaches ignore or are unable to capture. The underlying difference inthe frequency of binned expression levels is illustrated in FIG. 3, forthe textural phenotype τ, by bar graphs 310 and 312, and for spatialarrangement of the expression clusters,

, by bar graphs 320 and 322. The vectors 218 overlaid on the partitionedradiomic expression level maps 210 and 220 indicate the presence of anadjacent edge between two different expression level clusters. Note thatthe region of interest acquired from the patient with favorable responseillustrated in 210 has a higher proportion of medium to high expressionand more graph connections with high expression clusters than that ofthe patient with poor response illustrated in 220. This reflects thepresence of more spatially distinct treatment-related effects within thetumor region in patients with favorable response.

FIG. 5 illustrates radiomic feature heatmaps 510 and 512 ofrepresentative GBM patients with long-term survival (heatmap 510) orshort term survival (heatmap 512). Areas of high feature representationare illustrated at 516, and areas of low feature representation areillustrated at 514. The feature distributions in graph 520 showsignificant overlap between the distribution of short term survivalcurve 522 and long-term survival curve 524. In FIG. 6, the radiomicfeature illustrated in heatmaps 510 and 512 is partitioned into three(e.g., B=3) expression values and displayed as high (H) expression valueregion 616, medium (M) expression value region 614, and low (L)expression value region 612 illustrated in expression maps 610 and 620.The vectors 618 overlaid on the partitioned radiomic expression maps 610and 620 indicate the presence of an adjacent edge between two differentexpression level clusters.

FIG. 7 illustrates histograms 710, 712, 720, and 722 of the proportionof expression levels and adjacent connections between differentexpression levels which reveal underlying differences in the texturalphenotype (histograms 710 and 712), and spatial phenotype (histograms720 and 722), between the two prognostic outcomes respectively. In thisexample, radiomic heatmaps 510 and 512, and expression maps 610 and 620,are for the GLCM feature inertia, which is a measure of contrast withina neighborhood of pixels. The most prominent difference between thesetwo prognostic outcomes appears in the τ and

descriptors which indicate greater proportions of medium and highfrequency expression clusters and adjacent L-H expression levelconnections. The results indicate that more aggressive GBM cases (withpoor overall survival) have an imaging phenotype with more pronounced“hot spots”, which is not captured by the radiomic expressiondistributions graph 520.

FIG. 8 illustrates graphs 810 and 820 that demonstrate that RADISTATquantitatively outperforms the best statistic and texture (τ) for thehighest performing GLCM features including Inertia (p<0.001),Information Metric 1 (p<0.001), and Difference Variance (p<0.001).Graphs 810 and 820 further demonstrate that RADISTAT achieves higherAUCs than spatial (

) alone. FIG. 8 illustrates average AUCs across 25 runs of 3-fold crossvalidation for the RCa dataset illustrated in graph 810, and for the GBMdataset illustrated in graph 820, for the top three GLCM features ineach category. Embodiments described herein employing RADISTAT result ina consistently higher performance than any compared strategies,including the best performing statistical descriptor, the individualtextural, or the individual spatial components of RADISTAT. Thequantitative results illustrated in FIG. 8 demonstrate that RADISTATsignificantly outperforms top-ranked statistics for the three highestperforming GLCM features including energy, correlation, and differenceaverage (p<0.001 for each).

Embodiments described herein distinguish treatment response or predictoverall survival in patients demonstrating cancerous pathology withgreater accuracy than existing approaches that may only use a singlestatistic input to a predictive model. Embodiments more completelycharacterize the spatial diversity of over-expression andunder-expression exhibited by a radiomic feature, and further capturethe overall textural appearance of a radiomic feature based on therelative abundance of over-expression and under-expression, thusproviding greater discrimination between classes than existingapproaches. By increasing the accuracy with which treatment response oroverall survival in patients demonstrating cancerous pathology ispredicted, example methods and apparatus produce the concrete,real-world technical effect of increasing the probability that at-riskpatients receive timely treatment tailored to the particular pathologythey exhibit. The additional technical effect of reducing theexpenditure of resources and time on patients who have a less aggressivepathology is also achieved. Example embodiments further improve onexisting approaches by providing a more accurate second reader tofacilitate the reduction of inter-reader and intra-reader variabilityamong human radiologists, pathologists, or oncologists. Example methodsand apparatus thus improve on existing methods in a measurable,clinically significant way. When implemented as part of a personalizedmedicine system, a computer assisted diagnostic (CADx) system, atreatment response distinguishing system or an overall survivalprediction system, which may include a computer or a processorconfigured to predict treatment response or overall survival in patientsdemonstrating cancerous pathology, example embodiments improve theperformance of a machine, computer, or computer-related technology byproviding a more accurate and more reliable prediction of treatmentresponse or overall survival compared to existing approaches tocontrolling a machine to predict treatment response or overall survival.

Some portions of the detailed descriptions that follow are presented interms of algorithms and symbolic representations of operations on databits within a memory. These algorithmic descriptions and representationsare used by those skilled in the art to convey the substance of theirwork to others. An algorithm, here and generally, is conceived to be asequence of operations that produce a result. The operations may includephysical manipulations of physical quantities. Usually, though notnecessarily, the physical quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated in a logic, and so on. The physicalmanipulations create a concrete, tangible, useful, real-world result.

It has proven convenient at times, principally for reasons of commonusage, to refer to these signals as bits, values, elements, symbols,characters, terms, numbers, and so on. It should be borne in mind,however, that these and similar terms are to be associated with theappropriate physical quantities and are merely convenient labels appliedto these quantities. Unless specifically stated otherwise, it isappreciated that throughout the description, terms including processing,computing, calculating, determining, and so on, refer to actions andprocesses of a computer system, logic, circuit, processor, or similarelectronic device that manipulates and transforms data represented asphysical (electronic) quantities.

Example methods may be better appreciated with reference to flowdiagrams. While for purposes of simplicity of explanation, theillustrated methodologies are shown and described as a series of blocks,it is to be appreciated that the methodologies are not limited by theorder of the blocks, as some blocks can occur in different orders and/orconcurrently with other blocks from that shown and described. Moreover,less than all the illustrated blocks may be required to implement anexample methodology. Blocks may be combined or separated into multiplecomponents. Furthermore, additional and/or alternative methodologies canemploy additional, not illustrated blocks.

FIG. 9 is a flow diagram of an example set of operations 900 that may beperformed by a processor for predicting treatment response or overallsurvival in patients demonstrating cancerous pathology, including RCa orGBM. A processor(s) may include any combination of general-purposeprocessors and dedicated processors (e.g., graphics processors,application processors, etc.). The processors may be coupled with or mayinclude memory or storage and may be configured to execute instructionsstored in the memory or storage to enable various apparatus,applications, or operating systems to perform the operations. The memoryor storage devices may include main memory, disk storage, or anysuitable combination thereof. The memory or storage devices may include,but are not limited to any type of volatile or non-volatile memory suchas dynamic random access memory (DRAM), static random-access memory(SRAM), erasable programmable read-only memory (EPROM), electricallyerasable programmable read-only memory (EEPROM), Flash memory, orsolid-state storage.

The set of operations 900 includes, at 910, accessing a set of images ofa region of tissue demonstrating cancerous pathology. A member of theset of images includes a target tissue region of interest (ROI).Accessing the set of images includes acquiring electronic data, readingfrom a computer file, receiving a computer file, reading from a computermemory, or other computerized activity. A member of the set of imageshas a plurality of pixels, a pixel having an intensity. In oneembodiment, a member of the set of images is a 3 Tesla T2-w magneticresonance imaging (MRI) image of a region of tissue demonstrating rectalcancer (RCa), or a gadolinium-contrast (GD-C) T1-w MRI image of a regionof tissue demonstrating Glioblastoma multiforme (GBM).

In one embodiment, at 910, a member of the set of images demonstratingGBM is pre-processed by subjecting the member of the set of images tobias field correction, skull-stripping, and intensity standardization.In another embodiment, a member of the set of images is of a differentregion of tissue demonstrating a different form of cancerous pathology.In another embodiment, a member of the set of images is acquired using adifferent imaging technique, including computed tomography, or othertype of imaging technique.

The set of operations 900 also includes, at 912, extracting a set ofradiomic features from the ROI. A radiomic feature has a value. Thevalue may be based, at least in part, on a pixel intensity. In oneembodiment, the set of radiomic features includes at least one graylevel co-occurrence matrix (GLCM) feature. The at least one GLCM featuremay be an entropy feature, an energy feature, an inertia feature, acorrelation feature, an information measure 1 feature, an informationmeasure 2 feature, a sum average feature, a sum variance feature, a sumentropy feature, a difference average feature, a difference variancefeature, or a difference entropy feature. In one embodiment, the set ofradiomic features are extracted using a window size of three pixels inboth 2D and 3D. Using a window with, in this example, a size of threepixels, facilitates deriving the intensity co-occurrences in all spatialdirections. In another embodiment, the set of radiomic features mayinclude other types of radiomic feature, including, for example, a Gaborfeature, a co-occurrence of local anisotropic gradient tensors feature,a Laws feature, or other radiomic feature. The set of radiomic featuresincludes sub-visual features that cannot be perceived by the human eyeor extracted by pencil and paper. In one embodiment, the set of radiomicfeatures is extracted on a pixel-wise basis. For example, for each pixelin the ROI, a radiomic feature having a value may be extracted. Inanother example, for less than each pixel in the ROI, a radiomic featurehaving a value may be extracted.

The set of operations 900 also includes, at 920, defining a radiomicfeature expression scene based on the ROI and the set of radiomicfeatures. In one embodiment, the radiomic feature expression sceneincludes a spatial grid of pixels. A pixel in the grid of pixels isassociated with a radiomic feature value. In one embodiment, the set ofoperations further includes normalizing the radiomic feature expressionscene. Normalizing the radiomic feature expression scene may includenormalizing the radiomic feature expression scene from 0 to 1.

The set of operations 900 also includes, at 930, generating a clustermap by superpixel clustering the radiomic feature expression scene.Generating the cluster map by superpixel clustering the radiomic featureexpression scene includes using a modified simple linear iterativeclustering (SLIC) approach to generate K clusters. In this embodiment, Kis based on a minimum number of pixels in a cluster, and a distancebetween initial cluster seeds. K is an integer. In one embodiment, K isdefined based on the minimum number of pixels in a cluster and thedistance between initial cluster seeds. In one embodiment, the minimumnumber of pixels in a cluster is 7, and the distance between initialcluster seeds is 5. In one embodiment, the cluster map is normalizedsuch that an average radiomic feature value within a cluster has aminimum of 0 or a maximum of 1. In another embodiment, other parametersor techniques may be employed to superpixel cluster the radiomic featureexpression scene. For example, in one embodiment, α=5 and β=5 may beemployed when distinguishing responders from non-responders in RCa intwo dimensions, where a and p are values in pixels representing theminimum number of pixels in a cluster and the distance between initialcluster seeds respectively. In one embodiment, α=50 and β=7 may beemployed when distinguishing responders from non-responders in RCa inthree dimensions. In another embodiment, α=20 and β=5 may be employedwhen predicting overall survival in GBM in two dimensions. In anotherembodiment, α=100 and β=9 may be employed when predicting overallsurvival in GBM in three dimensions.

The set of operations 900 also includes, at 940, generating anexpression map by repartitioning the cluster map into B expressionlevels, where B is an integer. In one embodiment, B equals three. Inanother embodiment, B may have another, different value. B may beuser-selectable.

The set of operations 900 also includes, at 950, computing a texturalphenotype for the expression map based on the B expression levels.Computing the textural phenotype includes quantifying the fraction ofeach of B expression levels in the expression map. In one embodiment,computing the textural phenotype includes quantifying the fraction of athreshold number of B expression levels in the expression map. In oneembodiment, the textural phenotype is a 1×B feature vector.

The set of operations 900 also includes, at 952, computing a spatialphenotype for the expression map based on the B expression levels.Computing the spatial phenotype includes quantifying the adjacency foreach pairwise combination of B expression levels in the expression map.In one embodiment, computing the spatial phenotype includes quantifyingthe adjacency for a threshold number of pairwise combination of Bexpression levels in the expression map. In one embodiment, the spatialphenotype is a 1×N feature vector, where

$N = \begin{pmatrix}B \\2\end{pmatrix}$

is the total number of expression level pairs in the expression map.

The set of operations 900 also includes, at 960, constructing a radiomicspatial textural (RADISTAT) descriptor by combining the texturalphenotype with the spatial phenotype. In one embodiment, the RADISTATdescriptor is constructed by concatenating the textural phenotype withthe spatial phenotype to yield a 1×(B+N) vector. In another embodiment,the RADISTAT descriptor is constructed based on a weighted combinationof the textural phenotype and the spatial phenotype. In anotherembodiment, the RADISTAT descriptor is constructed based on a matrixmultiplication of the textural phenotype and the spatial phenotype. Inanother embodiment, the RADISTAT descriptor is constructed by othercombinations of the textural phenotype and the spatial phenotype.

The set of operations 900 also includes, at 970, providing the RADISTATdescriptor to a machine learning classifier. Providing the RADISTATdescriptor to a machine learning classifier includes acquiringelectronic data, reading from a computer file, receiving a computerfile, reading from a computer memory, or other computerized activity. Inone embodiment, the machine learning classifier is a linear discriminantanalysis (LDA) classifier. In another embodiment, the machine learningclassifier may be a, random forest classifier, a support vector machine(SVM) classifier, a convolutional neural network (CNN) classifier, orother type of machine learning or deep learning classifier.

The set of operations 900 also includes, at 980, receiving, from themachine learning classifier, a first probability that the ROI is aresponder or non-responder, or a second probability that the ROI willexperience long-term survival or short-term survival, where the machinelearning classifier computes the first probability or the secondprobability based, at least in part, on the RADISTAT descriptor.Receiving the first probability or the second probability from themachine learning classifier includes acquiring electronic data, readingfrom a computer file, receiving a computer file, reading from a computermemory, or other computerized activity.

The set of operations 900 also includes, at 990, generating aclassification of the ROI as a responder or non-responder, or as along-term survivor or short-term survivor based on the first probabilityor the second probability, respectively. For example, in one embodiment,the region of tissue may be classified as a responder if the machinelearning classifier provides a first probability greater than 0.5, whilethe region of tissue may be classified as a non-responder if the firstprobability is less than or equal to 0.5. In another embodiment, theregion of tissue may be classified as a responder if the firstprobability has another, different value, for example 0.6, 0.75, or 0.9.In one embodiment, the classification may be based on the firstprobability and at least one of the set of images, or the RADISTATdescriptor. In another embodiment, the region of tissue may beclassified as a long-term survivor if the machine learning classifierprovides a second probability greater than 0.5, while the region oftissue may be classified as a short-term survivor if the secondprobability is less than or equal to 0.5. In another embodiment, theregion of tissue may be classified as a long-term survivor if the secondprobability has another, different value, for example 0.6, 0.75, or 0.9.In embodiments described herein, short-term survival is defined as anoverall survival (OS) time of less than or equal to seven months, andlong term survival is defined as an OS greater than or equal to sixteenmonths. In other embodiments, other OS values may be employed. In oneembodiment, the classification may be based on the second probabilityand at least one of the set of images, or the RADISTAT descriptor.

In one embodiment, the operations 900 further include training themachine learning classifier. In this embodiment, the machine learningclassifier is trained and tested using a training set of images and atesting set of images. Training the machine learning classifier mayinclude training the machine learning classifier until a threshold levelof accuracy is achieved, until a threshold time has been spent trainingthe machine learning classifier, until a threshold amount ofcomputational resources have been expended training the machine learningclassifier, or until a user terminates training. Other trainingtermination conditions may be employed. Training the machine learningclassifier may also include determining which radiomic features andassociated RADISTAT descriptor are most discriminative in distinguishingresponders from non-responders, or predicting overall survival.

In one embodiment, a member of the set of images is acquired usingcontrast enhanced MRI. In this embodiment, the set of radiomic featuresdescribes the organization of contrast uptake on MRI. The modified SLICalgorithm is applied to cluster the pixels into sub-compartments basedon their contrast uptake profiles. Adjacent clusters are then mergedbased on similarity in average uptake profiles. A textural phenotypeexpressed as a 1×B vector T is obtained as described with respect toEquation 1 above. A spatial phenotype expressed as a 1×N vector f isobtained as

described with respect to Equation 3 above. The RADISTAT descriptor maythen be constructed from τ and

.

FIG. 10 illustrates an example set of operations 1000 that is similar tooperations 900 but that includes additional details and elements. Theset of operations 1000 further includes, at 1010, displaying theclassification and at least one of the first probability, the secondprobability, the RADISTAT descriptor, or the image. The set ofoperations 1000 further includes, at 1020, generating a personalizedcancer treatment plan based, at least in part, on the classification andat least one of the first probability, the second probability, theRADISTAT descriptor, or the image. Generating a personalized cancertreatment plan facilitates delivering a particular treatment that willbe therapeutically active to the patient, while minimizing negative oradverse effects experienced by the patient. For example, the cancertreatment plan may suggest a surgical treatment, may define animmunotherapy agent dosage or schedule, or a chemotherapy agent dosageor schedule, for a patient identified as likely to be a long-termsurvivor, or identified as likely to be a responder. For a patientclassified as likely to be a non-responder, or classified as ashort-term survivor, other treatments may be suggested.

FIG. 11 illustrates an example apparatus 1100 for predicting treatmentresponse or overall survival in cancerous pathology. Apparatus 1100includes a processor 1110. Apparatus 1100 also includes a memory 1120.Processor 1110 may, in one embodiment, include circuitry such as, butnot limited to, one or more single-core or multi-core processors.Processor 1110 may include any combination of general-purpose processorsand dedicated processors (e.g., graphics processors, applicationprocessors, etc.). The processors may be coupled with or may includememory (e.g. memory 1120) or storage and may be configured to executeinstructions stored in the memory or storage to enable variousapparatus, applications, or operating systems to perform the operations.Memory 1120 is configured to store a digitized image of a region oftissue demonstrating cancerous pathology.

In one embodiment, memory 1120 is configured to store a set of digitizedimages of a region of tissue demonstrating cancerous pathology. A memberof the set of digitized images includes a plurality of pixels, a pixelhaving an intensity.

Apparatus 1100 also includes an input/output (I/O) interface 1130.Apparatus 1100 also includes a set of circuits 1150. The set of circuits1150 includes an image acquisition circuit 1151, a radiomic featureexpression scene circuit 1152, a superpixel clustering circuit 1154, anexpression level mapping circuit 1156, a RADISTAT circuit 1157, and aclassification circuit 1159. Apparatus 1100 further includes aninterface 1140 that connects the processor 1110, the memory 1120, theI/O interface 1130, and the set of circuits 1150.

Image acquisition circuit 1151 is configured to access the set ofdigitized images of the region of tissue demonstrating cancerouspathology. In one embodiment, a member of the set of images may be a 3Tesla T2-w MRI image of a region of tissue demonstrating RCa, or a GD-CT1-w MRI image of a region of tissue demonstrating GBM. A member of theset of images may be a contrast enhanced MRI image. A member of the setof images includes a target tissue ROI. Accessing the set of imagesincludes acquiring electronic data, reading from a computer file,receiving a computer file, reading from a computer memory (e.g., memory1120), or other computerized activity. In another embodiment, accessingthe image may include accessing a network attached storage (NAS), acloud storage system, or other type of electronic storage system.Accessing the set of digitized images may, in one embodiment, includeaccessing a NAS device, a cloud storage system, or other type ofelectronic storage system using input/output interface 1130.

Radiomic feature expression scene circuit 1152 is configured to extracta set of radiomic features from the ROI. A radiomic feature has a value.In one embodiment, the set of radiomic features includes at least onegray level co-occurrence matrix (GLCM) feature. The at least one GLCMfeature may be an entropy feature, an energy feature, an inertiafeature, a correlation feature, an information measure 1 feature, aninformation measure 2 feature, a sum average feature, a sum variancefeature, a sum entropy feature, a difference average feature, adifference variance feature, or a difference entropy feature. In anotherembodiment, the set of radiomic features may include other types ofradiomic feature, including, for example, a Gabor feature, aco-occurrence of local anisotropic gradient orientations feature, a Lawsfeature, or other radiomic feature. In one embodiment, the set ofradiomic features is extracted on a pixel-wise basis. The set ofradiomic features are sub-visual features that cannot be perceived bythe human eye or extracted by pencil and paper. Radiomic featureexpression scene circuit 1152 is also configured to define a radiomicfeature expression scene based on the ROI and the set of radiomicfeatures. The radiomic feature expression scene includes a spatial gridof pixels, where a pixel in the grid of pixels is associated with aradiomic feature value. In one embodiment, radiomic feature expressionscene circuit 1152 is further configured to normalize the radiomicfeature expression scene.

Superpixel clustering circuit 1154 is configured to generate a clustermap based on the image and the set of radiomic features. Superpixelclustering circuit 1154 generates the cluster map by superpixelclustering the radiomic feature expression scene. In one embodiment,superpixel clustering circuit 1154 is configured to superpixel clusterthe radiomic feature expression scene using a modified simple lineariterative clustering (SLIC) approach to generate K clusters. K is basedon a minimum number of pixels in a cluster, and a distance betweeninitial cluster seeds. K is an integer. In one embodiment, superpixelclustering circuit 1154 is further configured to normalize the clustermap such that an average radiomic feature value within a cluster has aminimum of 0 or a maximum of 1.

Expression level mapping circuit 1156 is configured to generate anexpression map based on the cluster map. Expression level mappingcircuit 1156 generates the expression map by repartitioning the clustermap into B expression levels. B is an integer. In one embodiment, Bequals 3. In another embodiment, B may have another, different value(e.g., 4, 5, 7). In one embodiment, B may be user selectable oradjustable. B may be selected or adjusted based, for example, on imagesize, image resolution, image quality, experimental results, theparticular pathology being investigated, computational resources,processing time, or other consideration.

RADISTAT circuit 1157 is configured to compute a textural phenotype forthe expression map based on the B expression levels. RADISTAT circuit1157 is also configured compute a spatial phenotype for the expressionmap based on the B expression levels. Computing the textural phenotypeincludes quantifying the fraction of each of B expression levels in theexpression map. Computing the spatial phenotype includes quantifying theadjacency for each pairwise combination of B expression levels in theexpression map. In one embodiment, computing the textural phenotypeincludes quantifying the fraction of a threshold number of B expressionlevels in the expression map. In one embodiment, computing the spatialphenotype includes quantifying the adjacency for a threshold number ofpairwise combination of B expression levels in the expression map.

RADISTAT circuit 1157 is also configured to construct the RADISTATdescriptor by concatenating the textural phenotype with the spatialphenotype. In another embodiment, RADISTAT circuit 1157 is configured toconstruct the RADISTAT descriptor based on a weighted combination of thetextural phenotype and the spatial phenotype. In another embodiment,RADISTAT circuit 1157 is configured to construct the RADISTAT descriptorbased on a matrix multiplication of the textural phenotype and thespatial phenotype. In another embodiment, the RADISTAT descriptor mayuse a different combination of phenotypes. RADISTAT circuit 1157 isfurther configured to compute a first probability that the ROI is aresponder or non-responder, based, at least in part, on the RADISTATdescriptor. RADISTAT circuit 1157 is further configured to compute asecond probability that the ROI will experience long-term survival orshort-term survival, based, at least in part, on the RADISTATdescriptor.

In one embodiment, RADISTAT circuit 1157 includes a machine learningclassifier or includes machine learning classifier circuitry configuredto compute the first probability or the second probability using an LDAclassification approach. In another embodiment, RADISTAT circuit 1157 isconfigured as another different type of machine learning classifier,including a support vector machine (SVM), a random forest classifier, aquadratic discriminant analysis (QDA) classifier, a convolutional neuralnetwork (CNN), or other type of machine learning or deep learningclassifier.

Classification circuit 1159 is configured to classify the ROI as aresponder or non-responder, or as long-term survivor or short-termsurvivor based, at least in part, on the first probability or the secondprobability, respectively. While two classes are described here (e.g.,responder/non-responder, or long-term survivor/short-term survivor),other classifications (e.g., responder/unknown/non-responder) may beemployed.

FIG. 12 illustrates an example apparatus 1200 that is similar toapparatus 1100 but that includes additional elements and details. In oneembodiment, apparatus 1200 further includes a treatment plan generationcircuit 1258 configured to generate a treatment plan based, at least inpart, on the classification. In one embodiment, treatment plangeneration circuit generates the treatment plan based on theclassification and at least one of the RADISTAT descriptor, the imagethe first probability, or the second probability. In one embodiment,apparatus 1200 further includes a display circuit 1259 configured todisplay the treatment plan, the image, the RADISTAT descriptor, thefirst probability, the second probability, or the classification.

Treatment plan generation circuit 1258 is configured to generate acancer treatment plan for the patient of which the set of digitizedimages was acquired based, at least in part, the classification, the setof digitized images, the RADISTAT descriptor, the first probability, orthe second probability. Defining a personalized cancer treatment planfacilitates delivering a particular treatment that will betherapeutically active to the patient, while minimizing negative oradverse effects experienced by the patient. For example, the cancertreatment plan may suggest a surgical treatment, may define animmunotherapy agent dosage or schedule, or a chemotherapy agent dosageor schedule, for a patient identified as likely to be a long-termsurvivor, or identified as likely to be a responder. For a patientclassified as likely to be a non-responder, or classified as ashort-term survivor, other treatments may be suggested.

In another embodiment, apparatus 1100 or apparatus 1200 may control aCADx system to classify the region of tissue represented in the image,at least in part, on the first probability or the second probability. Inother embodiments, other types of CADx systems may be controlled,including CADx systems for predicting treatment response or overallsurvival time in other tissue presenting other, different pathologiesthat may be distinguished based on a RADISTAT descriptor generated fromfeatures extracted from other types of radiological images. For example,embodiments described herein may be employed to predict treatmentresponse or overall survival time using a machine learning classifier inbreast cancer (BCa), kidney disease, lung cancer, or prostate cancer.

Display circuit 1259 is configured to display the treatment plan, theimage, the RADISTAT descriptor, the first probability, the secondprobability, or the classification. In one embodiment, display circuit1259 is configured to display the treatment plan, the image, theRADISTAT descriptor, the first probability, the second probability, orthe classification on a computer monitor, a smartphone display, a tabletdisplay, or other displays. Displaying the treatment plan, the image,the RADISTAT descriptor, the first probability, the second probability,or the classification may also include printing the treatment plan, theimage, the RADISTAT descriptor, the first probability, the secondprobability, or the classification. Display circuit 1259 may alsocontrol a CADx system, a monitor, or other display, to display operatingparameters or characteristics of image acquisition circuit 1151,radiomic feature expression scene circuit 1152, superpixel clusteringcircuit 1154, expression level mapping circuit 1156, RADISTAT circuit1157, or classification circuit 1159, including a machine learningclassifier, during both training and testing, or during clinicaloperation of apparatus 1100 or apparatus 1200.

In another embodiment of apparatus 1100 or 1200, the set of circuits1150 further includes a training circuit configured to train RADISTATcircuit 1157. Training RADISTAT circuit 1157 may include training amachine learning classifier. In one embodiment, the training circuit isconfigured to access a dataset of digitized images of a region ofinterest demonstrating RCa or GBM. In this embodiment, the machinelearning classifier is trained and tested using a training set of imagesand a testing set of images. Training the machine learning classifiermay include training the machine learning classifier until a thresholdlevel of accuracy is achieved, until a threshold time has been spenttraining the machine learning classifier, until a threshold amount ofcomputational resources have been expended training the machine learningclassifier, or until a user terminates training. Other trainingtermination conditions may be employed.

FIG. 12 also illustrates a personalized medicine device 1260.Personalized medicine device 1260 may be, for example, a CADx system, aRCa or GBM overall survival time prediction system, a RCa or GBMtreatment response prediction system, or other type of personalizedmedicine device that may be used to facilitate the prediction oftreatment response or survival time. In one embodiment, treatment plangeneration circuit 1258 may control personalized medicine device 1260 todisplay the treatment plan, the image, the RADISTAT descriptor, thefirst probability, the second probability, or the classification on acomputer monitor, a smartphone display, a tablet display, or otherdisplays.

Embodiments described herein, including at least apparatus 1100 and1200, resolve features extracted from the set of digitized images at ahigher order or higher level than a human can resolve in the human mindor with pencil and paper. For example, the GLCM features are notbiological properties of cancerous tissue that a human eye can perceive.A tumor does not include a GLCM, and these features cannot be stored ina human mind. The RADISTAT descriptor provided to the machine learningclassifier is of a different nature than the tumor represented in theimage, or the GLCM features. The probabilities computed by RADISTATcircuit 1157 are of a fundamentally different nature than the set ofdigitized images, or of the tissue from which the images were generated.

Displaying the treatment plan, the image, the RADISTAT descriptor, thefirst probability, the second probability, or the classificationinvolves but is not limited to extracting and changing the character ofinformation present in a region of tissue (e.g. biological tissue), to aradiological image (e.g. MRI image), to changing the information presentin the image to information of a different character in the set ofradiomics features, the probability, and the treatment plan. Embodimentsdescribed herein further transform the character of information toinformation suitable for display on, for example, a computer monitor, asmartphone display, a tablet display, or other displays. Thus,embodiments described herein use a combined order of specific rules,elements, or components that render information into a specific formatthat is then used and applied to create desired results more accurately,more consistently, and with greater reliability than existingapproaches.

FIG. 14 illustrates a computerized method 1400 for characterizingintra-tumoral heterogeneity for treatment response and outcomeprediction in cancerous pathology. Method 1400 may, in one embodiment,be implemented by apparatus 1100 or apparatus 1200, or computer 1300.Method 1400 includes, at 1410 accessing a set of digitized images of aregion of tissue demonstrating cancerous pathology. In one embodiment, amember of the set of digitized images may be a 2T-w MRI image of aregion of tissue demonstrating RCa, a GD-C T1-w MRI image of a region oftissue demonstrating GBM, a contrast enhanced MRI image of a region oftissue demonstrating cancerous pathology, or a radiological image of aregion of tissue demonstrating other, cancerous pathology. Accessing theset of images includes acquiring electronic data, reading from acomputer file, receiving a computer file, reading from a computermemory, or other computerized activity. A member of the set of imageshas a plurality of pixels, a pixel having an intensity.

Method 1400 also includes, at 1420 defining an input radiomic scene Ibased on a member of the set of images. The input radiomic scene I mayinclude a spatial grid of pixels, where a pixel is associated with aradiomic feature value. Defining the input radiomic scene I may furtherinclude normalizing the range of the input radiomic scene I.

Method 1400 also includes, at 1430, generating a cluster map Î. Thecluster map Î is generated by applying superpixel clustering to theinput radiomic scene I. In one embodiment, a modified SLIC approach isemployed to generate clusters. In one embodiment, generating the clustermap Î may further include normalizing cluster map Î.

Method 1400 also includes, at 1440, generating an expression map Ĩ byrepartitioning cluster map Î into B expression levels. B is an integer.In one embodiment where B=3, a first expression level indicates low (L)feature expression, a second expression level indicates medium (M)expression, and a third expression level indicates high (H) featureexpression. In another embodiment, B may have another different value(e.g., B=4, B=5), and expression map Ĩ may be generated byrepartitioning cluster map Î into the other, different number ofexpression levels.

Method 1400 also includes, at 1450, computing a textural phenotype. Thetextural phenotype is computed as the proportion of each of the Bexpression levels in the expression map Ĩ. Thus, when B=3, the texturalphenotype is computed as the proportion of each of the low, medium, andhigh expression levels in the expression map Ĩ. The textural phenotypemay be expressed as a 1×B feature vector.

Method 1400 also includes, at 1460, computing a spatial phenotype. Thespatial phenotype is computed as the number of times L-M, M-H, and L-Hexpression levels are adjacent to each other, based on adjacencyrepresented in the expression map Ĩ. The spatial phenotype may beexpressed as a 1×N feature vector, where

$N = \begin{pmatrix}B \\2\end{pmatrix}$

is the total number of expression level pairs in the expression map.

Method 1400 also includes, at 1470, generating a RADISTAT descriptor byconcatenating the textural phenotype with the spatial phenotype. In oneembodiment, the RADISTAT descriptor characterizes the spatial diversityof feature over-expression and feature under-expression exhibited in theexpression map.

Method 1400 further includes, at 1480, generating a classification ofthe region of tissue as a responder or non-responder, or as a long-termsurvivor or short-term survivor based, at least in part, on the RADISTATdescriptor. In one embodiment, method 1400 further includes displayingthe classification or the RADISTAT descriptor on a computer monitor,smartphone display, or other type of electronic display device.

Improved identification or classification of patients who will respondor not respond to treatment, or who will be long-term survivors or shortterm survivors may produce the technical effect of improving treatmentefficacy by increasing the accuracy of and decreasing the time requiredto treat patients demonstrating RCa or GBM, or other forms of cancerouspathology. Treatments and resources, including expensive immunotherapyagents or chemotherapy may be more accurately tailored to patients witha likelihood of benefiting from said treatments and resources, includingresponding to immunotherapy or chemotherapy, so that more appropriatetreatment protocols may be employed, and expensive resources are notwasted. Controlling a personalized medicine system, a CADx system, aprocessor, or a RCa or GBM response or survival time prediction systembased on improved, more accurate identification or classification ofpatients who will experience RCa or GBM further improves the operationof the system, processor, or apparatus, since the accuracy of thesystem, processor, or apparatus is increased and unnecessary operationswill not be performed.

Using a more appropriately modulated treatment may lead to lessaggressive therapeutics being required for a patient or may lead toavoiding or delaying a biopsy, a resection, or other invasive procedure.When patients experiencing RCa or GBM who will more likely experiencetreatment response or long-term survival are more quickly and moreaccurately distinguished from patients who will not, patients most atrisk may receive a higher proportion of scarce resources (e.g.,therapeutics, physician time and attention, hospital beds) while thoseless likely to benefit from the treatment may be spared unnecessarytreatment, which in turn spares unnecessary expenditures and resourceconsumption. Example methods, apparatus, and other embodiments may thushave the additional effect of improving patient outcomes compared toexisting approaches.

While FIGS. 4, 9-10, and 14 illustrate various actions occurring inserial, it is to be appreciated that various actions illustrated inFIGS. 4, 9-10, and 14 could occur substantially in parallel. By way ofillustration, a first process could involve computing a texturalphenotype, a second process could involve computing a spatial phenotype,and a third process could involve classifying a region of interest.While three processes are described, it is to be appreciated that agreater or lesser number of processes could be employed and thatlightweight processes, regular processes, threads, and other approachescould be employed.

In one example, a method may be implemented as computer executableinstructions. Thus, in one example, a computer-readable storage devicemay store computer executable instructions that if executed by a machine(e.g., computer, processor) cause the machine to perform methods oroperations described or claimed herein including methods or operations900, 1000, or 1400. While executable instructions associated with thelisted methods are described as being stored on a computer-readablestorage device, it is to be appreciated that executable instructionsassociated with other example methods described or claimed herein mayalso be stored on a computer-readable storage device. In differentembodiments the example methods described herein may be triggered indifferent ways. In one embodiment, a method may be triggered manually bya user. In another example, a method may be triggered automatically.

FIG. 13 illustrates an example computer 1300 in which example methods oroperations illustrated herein can operate and in which example methods,apparatus, circuits, operations, or logics may be implemented. Indifferent examples, computer 1300 may be part of a personalized medicinesystem, an RCa or GBM treatment response or survival time predictionsystem, an MRI system, a digital whole slide scanner, a CT system, maybe operably connectable to a CT system, an MRI system, a personalizedmedicine system, or a digital whole slide scanner, or may be part of aCADx system.

Computer 1300 includes a processor 1302, a memory 1304, and input/output(I/O) ports 1310 operably connected by a bus 1308. In one example,computer 1300 may include a set of logics or circuits 1330 that performa method of predicting treatment response or survival time using amachine learning classifier. Thus, the set of circuits 1330, whetherimplemented in computer 1300 as hardware, firmware, software, and/or acombination thereof may provide means (e.g., hardware, firmware,circuits) for characterizing intra-tumoral heterogeneity, or predictingtreatment response or survival time of cancer patients, including RCa orGBM patients. In different examples, the set of circuits 1330 may bepermanently and/or removably attached to computer 1300.

Processor 1302 can be a variety of various processors including dualmicroprocessor and other multi-processor architectures. Processor 1302may be configured to perform steps of methods claimed and describedherein. Memory 1304 can include volatile memory and/or non-volatilememory. A disk 1306 may be operably connected to computer 1300 via, forexample, an input/output interface (e.g., card, device) 1318 and aninput/output port 1310. Disk 1306 may include, but is not limited to,devices like a magnetic disk drive, a tape drive, a Zip drive, a flashmemory card, or a memory stick. Furthermore, disk 1306 may includeoptical drives like a CD-ROM or a digital video ROM drive (DVD ROM).Memory 1304 can store processes 1314 or data 1317, for example. Data1317 may, in one embodiment, include digitized MRI images of a region oftissue demonstrating RCa or GBM. Disk 1306 or memory 1304 can store anoperating system that controls and allocates resources of computer 1300.

Bus 1308 can be a single internal bus interconnect architecture or otherbus or mesh architectures. While a single bus is illustrated, it is tobe appreciated that computer 1300 may communicate with various devices,circuits, logics, and peripherals using other buses that are notillustrated (e.g., PCIE, SATA, Infiniband, 1394, USB, Ethernet).

Computer 1300 may interact with input/output devices via I/O interfaces1318 and input/output ports 1310. Input/output devices can include, butare not limited to, CT systems, MRI systems, digital whole slidescanners, an optical microscope, a keyboard, a microphone, a pointingand selection device, cameras, video cards, displays, disk 1306, networkdevices 1320, or other devices. Input/output ports 1310 can include butare not limited to, serial ports, parallel ports, or USB ports.

Computer 1300 may operate in a network environment and thus may beconnected to network devices 1320 via I/O interfaces 1318 or I/O ports1310. Through the network devices 1320, computer 1300 may interact witha network. Through the network, computer 1300 may be logically connectedto remote computers. The networks with which computer 1300 may interactinclude, but are not limited to, a local area network (LAN), a wide areanetwork (WAN), or other networks, including the cloud.

Examples herein can include subject matter such as an apparatus, apersonalized medicine system, a CADx system, a processor, a system, amethod, means for performing acts, steps, or blocks of the method, atleast one machine-readable medium including executable instructionsthat, when performed by a machine (e.g., a processor with memory, anapplication-specific integrated circuit (ASIC), a field programmablegate array (FPGA), or the like) cause the machine to perform acts of themethod or of an apparatus or system for predicting cancer treatmentresponse or survival time, according to embodiments and examplesdescribed.

References to “one embodiment”, “an embodiment”, “one example”, and “anexample” indicate that the embodiment(s) or example(s) so described mayinclude a particular feature, structure, characteristic, property,element, or limitation, but that not every embodiment or examplenecessarily includes that particular feature, structure, characteristic,property, element or limitation. Furthermore, repeated use of the phrase“in one embodiment” does not necessarily refer to the same embodiment,though it may.

“Computer-readable storage device”, as used herein, refers to a devicethat stores instructions or data. “Computer-readable storage device”does not refer to propagated signals. A computer-readable storage devicemay take forms, including, but not limited to, non-volatile media, andvolatile media. Non-volatile media may include, for example, opticaldisks, magnetic disks, tapes, and other media. Volatile media mayinclude, for example, semiconductor memories, dynamic memory, and othermedia. Common forms of a computer-readable storage device may include,but are not limited to, a floppy disk, a flexible disk, a hard disk, amagnetic tape, other magnetic medium, an application specific integratedcircuit (ASIC), a compact disk (CD), other optical medium, a randomaccess memory (RAM), a read only memory (ROM), a memory chip or card, amemory stick, and other media from which a computer, a processor orother electronic device can read.

“Circuit”, as used herein, includes but is not limited to hardware,firmware, software in execution on a machine, or combinations of each toperform a function(s) or an action(s), or to cause a function or actionfrom another logic, method, or system. A circuit may include a softwarecontrolled microprocessor, a discrete logic (e.g., ASIC), an analogcircuit, a digital circuit, a programmed logic device, a memory devicecontaining instructions, and other physical devices. A circuit mayinclude one or more gates, combinations of gates, or other circuitcomponents. Where multiple logical circuits are described, it may bepossible to incorporate the multiple logical circuits into one physicalcircuit. Similarly, where a single logical circuit is described, it maybe possible to distribute that single logical circuit between multiplephysical circuits.

To the extent that the term “includes” or “including” is employed in thedetailed description or the claims, it is intended to be inclusive in amanner similar to the term “comprising” as that term is interpreted whenemployed as a transitional word in a claim.

Throughout this specification and the claims that follow, unless thecontext requires otherwise, the words ‘comprise’ and ‘include’ andvariations such as ‘comprising’ and ‘including’ will be understood to beterms of inclusion and not exclusion. For example, when such terms areused to refer to a stated integer or group of integers, such terms donot imply the exclusion of any other integer or group of integers.

To the extent that the term “or” is employed in the detailed descriptionor claims (e.g., A or B) it is intended to mean “A or B or both”. Whenthe applicants intend to indicate “only A or B but not both” then theterm “only A or B but not both” will be employed. Thus, use of the term“or” herein is the inclusive, and not the exclusive use. See, Bryan A.Garner, A Dictionary of Modern Legal Usage 624 (2d. Ed. 1995).

While example systems, methods, and other embodiments have beenillustrated by describing examples, and while the examples have beendescribed in considerable detail, it is not the intention of theapplicants to restrict or in any way limit the scope of the appendedclaims to such detail. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the systems, methods, and other embodiments described herein.Therefore, the invention is not limited to the specific details, therepresentative apparatus, and illustrative examples shown and described.Thus, this application is intended to embrace alterations,modifications, and variations that fall within the scope of the appendedclaims.

What is claimed is:
 1. A non-transitory computer-readable storage devicestoring computer executable instructions that when executed control aprocessor to perform operations, the operations including: accessing animage of a region of interest (ROI) demonstrating cancerous pathology,the image having a pixel, a pixel having an intensity; extracting a setof radiomic features from the ROI, a radiomic feature having a value;defining a radiomic feature expression scene based on the ROI and theset of radiomic features; generating a cluster map by superpixelclustering the radiomic feature expression scene; generating anexpression map by repartitioning the cluster map into B expressionlevels, where B is an integer; computing a textural phenotype for theexpression map based on the B expression levels; computing a spatialphenotype for the expression map based on the B expression levels;constructing a radiomic spatial textural (RADISTAT) descriptor byconcatenating the textural phenotype with the spatial phenotype;providing the RADISTAT descriptor to a machine learning classifier;receiving, from the machine learning classifier, a first probabilitythat the ROI is a responder or non-responder, or a second probabilitythat the ROI will experience long-term survival or short-term survival,where the machine learning classifier computes the first probability orthe second probability based, at least in part, on the RADISTATdescriptor; and generating a classification of the ROI as a responder ornon-responder, or as a long-term survivor or short-term survivor basedon the first probability or the second probability, respectively.
 2. Thenon-transitory computer-readable storage device of claim 1, where theset of radiomic features includes at least one gray level co-occurrencematrix (GLCM) feature, where the at least one GLCM feature is an entropyfeature, an energy feature, an inertia feature, a correlation feature,an information measure 1 feature, an information measure 2 feature, asum average feature, a sum variance feature, a sum entropy feature, adifference average feature, a difference variance feature, or adifference entropy feature, or where the set of radiomic featuresincludes a Gabor feature, a co-occurrence of local anisotropic gradienttensors feature, or a Laws feature.
 3. The non-transitorycomputer-readable storage device of claim 2, where the set of radiomicfeatures is extracted on a pixel-wise basis.
 4. The non-transitorycomputer-readable storage device of claim 1, where the radiomic featureexpression scene includes a spatial grid of pixels, where a pixel in thegrid of pixels is associated with a radiomic feature value.
 5. Thenon-transitory computer-readable storage device of claim 4, where therange of radiomic feature expression scene is normalized to lie between0 and
 1. 6. The non-transitory computer-readable storage device of claim1, where generating the cluster map by superpixel clustering theradiomic feature expression scene includes using a modified simplelinear iterative clustering (SLIC) approach to generate K clusters,where K is based on a minimum number of pixels in a cluster, and adistance between initial cluster seeds, where K is an integer.
 7. Thenon-transitory computer-readable storage device of claim 6, where thecluster map is normalized such that an average radiomic feature valuewithin a cluster has a minimum of 0 or a maximum of
 1. 8. Thenon-transitory computer-readable storage device of claim 6, where Bequals 3, the minimum number of pixels in a cluster is 7, and thedistance between initial cluster seeds is 5 pixels.
 9. Thenon-transitory computer-readable storage device of claim 1, wherecomputing the textural phenotype includes quantifying the fraction ofeach of B expression levels in the expression map.
 10. Thenon-transitory computer-readable storage device of claim 1, wherecomputing the spatial phenotype includes quantifying the adjacency foreach pairwise combination of B expression levels in the expression map.11. The non-transitory computer-readable storage device of claim 1, theoperations further comprising displaying the classification and at leastone of the first probability, the second probability, the RADISTATdescriptor, or the image.
 12. The non-transitory computer-readablestorage device of claim 1, the operations further comprising generatinga personalized cancer treatment plan based, at least in part, on theclassification and at least one of the first probability, the secondprobability, the RADISTAT descriptor, or the image.
 13. Thenon-transitory computer-readable storage device of claim 1, where theimage is a 3 Tesla T2-w magnetic resonance imaging (MRI) image of aregion of tissue demonstrating rectal cancer (RCa), agadolinium-contrast (GD-C) T1-w MRI image of a region of tissuedemonstrating Glioblastoma multiforme (GBM), a contrast enhanced MRIimage of a region of tissue demonstrating cancerous pathology, or adiffusion-weighted MRI image of a region of tissue demonstratingcancerous pathology.
 14. An apparatus for predicting treatment responseor survival time in cancerous pathology, the apparatus comprising: aprocessor; a memory configured to store a set of digitized images of aregion of interest (ROI) demonstrating cancerous pathology, where amember of set of digitized images has a plurality of pixels, a pixelhaving an intensity; an input/output (I/O) interface; a set of circuitscomprising an image acquisition circuit, a radiomic feature expressionscene circuit, a superpixel clustering circuit, an expression levelmapping circuit, a radiomic spatial textural descriptor (RADISTAT)circuit, and a classification circuit; and an interface that connectsthe processor, the memory, the I/O interface, and the set of circuits;the image acquisition circuit configured to access a member of set ofdigitized images of the ROI demonstrating cancerous pathology; theradiomic feature expression scene circuit configured: extract a set ofradiomic features from the ROI, a radiomic feature having a value; anddefine a radiomic feature expression scene based on the ROI and the setof radiomic features; the superpixel clustering circuit configured togenerate a cluster map by superpixel clustering the radiomic featureexpression scene; the expression level mapping circuit configured togenerate an expression map by repartitioning the cluster map into Bexpression levels, where B is an integer; the RADISTAT circuitconfigured to: compute a textural phenotype for the expression map basedon the B expression levels; compute a spatial phenotype for theexpression map based on the B expression levels; construct a RADISTATdescriptor by concatenating the textural phenotype with the spatialphenotype; and compute a first probability that the ROI is a responderor non-responder, or a second probability that the ROI will experiencelong-term survival or short-term survival, based, at least in part, onthe RADISTAT descriptor; and the classification circuit configured togenerate a classification of the ROI as a responder or non-responder, oras long-term survivor or short-term survivor based, at least in part, onthe first probability or the second probability, respectively.
 15. Theapparatus of claim 14, where the radiomic feature expression sceneincludes a spatial grid of pixels, where a pixel in the grid of pixelsis associated with a radiomic feature value.
 16. The apparatus of claim14, where the superpixel clustering circuit is configured to generatethe cluster map by superpixel clustering the radiomic feature expressionscene using a modified simple linear iterative clustering (SLIC)approach to generate K clusters, where K is based on a minimum number ofpixels in a cluster, and a distance between initial cluster seeds, whereK is an integer, and where the superpixel clustering circuit isconfigured to normalize the cluster map such that an average radiomicfeature value within a cluster has a minimum of 0 or a maximum of
 1. 17.The apparatus of claim 14, where B=3.
 18. The apparatus of claim 14,where the RADISTAT circuit is configured to: compute the texturalphenotype for the expression map based on the B expression levels byquantifying the fraction of each of B expression levels in theexpression map; and compute the spatial phenotype for the expression mapbased on the B expression levels by quantifying the adjacency for eachpairwise combination of B expression levels in the expression map. 19.The apparatus of claim 15, the set of circuits further comprising a:display circuit configured to display the classification, the firstprobability, the second probability, the RADISTAT descriptor, the ROI,or the member of the set of images; and a cancer treatment plan circuitconfigured to generate a personalized cancer treatment plan based, atleast in part, on the classification and at least one of the firstprobability, the second probability, the RADISTAT descriptor, or themember of the set of images.
 20. A non-transitory computer-readablestorage device storing computer-executable instructions that whenexecuted by a computer control the computer to perform a methodcomprising: accessing a set of digitized images of a region of tissuedemonstrating cancerous pathology, where a member of the set of imageshas a plurality of pixels, a pixel having an intensity; defining aninput radiomic scene I based on a member of the set of images, where theinput radiomic scene I includes a spatial grid of pixels, where a pixelis associated with a radiomic feature value; generating a cluster map Îby applying superpixel clustering to the input radiomic scene I;generating an expression map Ĩ by repartitioning cluster map Î into Bexpression levels, where B is an integer; computing a textural phenotypebased on the proportion of each of B expression levels in the expressionmap Ĩ; computing a spatial phenotype based on the adjacency of the Bexpression levels represented in the expression map Ĩ; generating aRADISTAT descriptor by concatenating the textural phenotype with thespatial phenotype; and generating a classification of the region oftissue as a responder or non-responder, or as a long-term survivor orshort-term survivor based, at least in part, on the RADISTAT descriptor.